The invention relates generally to the field of image processing, and in particular to the field of image assessment and understanding.
Image assessment and understanding deal with problems that are easily solved by human beings given their intellectual faculties but are extremely difficult to solve by fully automated computer systems. Image understanding problems that are considered important in photographic applications include main subject detection, scene classification, sky and grass detection, people detection, automatic detection of orientation, etc. In a variety of applications that deal with a group of pictures, it is important to rank the images in terms of a logical order, so that they can be processed or treated according to their order. A photographic application of interest is automatic albuming, where a group of digital images are automatically organized into digital photo albums. This involves clustering the images into separate events and then laying out each event in some logical order, if possible. This order implies at least some attention to the relative content of the images, i.e., based on the belief that some images would likely be preferred over others.
A number of known algorithms, such as dud detection, event detection and page layout algorithms, are useful in connection with automatic albuming applications. Dud detection addresses the elimination, or de-emphasis, of duplicate images and poor quality images, while event detection involves the clustering of images into separate events by certain defined criteria, such as date and time. Given a set of images that belong to the same event, the objective of page layout is to layout each event in some logical and pleasing presentation, e.g., to find the most pleasing and space-efficient presentation of the images on each page. It would be desirable to be able to select the most important image in the group of images, e.g., the one that should receive the most attention in a page layout.
Due to the nature of the image assessment problem, i.e., that an automated system is expected to generate results that are representative of high-level cognitive human (understanding) processes, the design of an assessment system is a challenging task. Effort has been devoted to evaluating text and graphical data for its psychological effect, with the aim of creating or editing a document for a particular visual impression (see, e.g., U.S. Pat. Nos. 5,875,265 and 5,424,945). In the ""265 patent, a system analyzes an image, in some case with the aid of an operator, to determine correspondence of visual features to sensitive language that is displayed for use by the operator. The difficulty in this system is that the visual features are primarily based on low level features, i.e., color and texture, that are not necessarily related to image content, and a language description is difficult is to use for relative ranking of images. The ""945 patent discloses a system for evaluating the psychological effect of text and graphics in a document. The drawback with the ""945 patent is that it evaluates the overall visual impression of the document, without regard to its specific content, which reduces its usefulness for developing relative ranking. Besides their complexity and orientation toward discernment of a psychological effect, these systems focus on the analysis and creation of a perceptual impression rather than on the assessment and utilization of an existing image.
Determining the number of copies of prints in an order usually implies at least some judgment about the images. Some optical and digital printing systems currently available have the capability of printing more than one print per input image. Typically, a double print option is bundled at the point of sale, which results in two prints produced for each image received. Digital image processing algorithms which analyze the pixel values of an original digital image, or a digital image derived from a photographic film negative, are also included in some printing systems for the purposes, in some circumstances, of rejecting the image altogether. These types of algorithms evaluate an image and a decision is made to either print or not print the image. However, the same number of prints is made for all images which pass the evaluation test.
It is commonly understood that not all images from a set of images have the same esthetic value, and therefore may not be copied in the same number. Some working solutions for digital imaging applications have addressed this issue by including non-image pixel information with a digital image which indicates the number of prints requested. This information is then utilized by a digital printer to vary the number of prints made per image. Alternatively, a particular image within a set of images may be singled out manually by either the operator of the printing equipment or by the customer in order to specially enhance that particular image, making it more desirable and ultimately worthy of more copies. However, each of these techniques require user intervention. What is needed is an automatic digital imaging algorithm which can make an intelligent decision without user input as to the number of prints to be made.
The present invention is directed to overcoming one or more of the problems set forth above. Briefly summarized, according to one aspect of the present invention, a method is disclosed for varying one or more print attributes of a print made from a digital image, including the steps of (a) computing a print attribute value for the digital image based on a determination of the degree of importance, interest or attractiveness of the image, and (b) using the print attribute value to control a print attribute of the print made from the image. In a typical embodiment, the print attribute is either the number of prints made of the image, the size of a print made from the image, or the magnification factor used for the image.
From another perspective, the invention pertains to a method for varying one or more print attributes of a print made from a digital image composed of image pixels, including the steps of (a) computing a print attribute value for the digital image based on a determination derived from the image pixels; and (b) using the print attribute value to control either the number of prints made of the image, the size of a print made from the image, or the magnification factor used for the image.
In one embodiment, the print attribute value is based on an appeal value determined from the degree of importance, interest or attractiveness that is intrinsic to the image. In another embodiment, wherein the image is one of a group of digital images, the print attribute value is based on an emphasis value determined from the degree of importance, interest or attractiveness of the image relative to other images in the group of images.
The determination of the degree of importance, interest or attractiveness of an image is based on an assessment of the image with respect to certain features, wherein one or more quantities are computed that are related to one or more features in each digital image, including one or more features pertaining to the content of the individual digital image. The quantities are processed with a reasoning algorithm that is trained on the opinions of one or more human observers, and an output is obtained from the reasoning algorithm that assesses each image. In a dependent aspect of the invention, the features pertaining to the content of the digital image include either people-related features or subject-related features. Moreover, additional quantities may be computed that relate to one or more objective measures of the digital image, such as colorfulness or sharpness. The results of the reasoning algorithm are processed to rank order the quality of each image in the set of images. The print attributes applied to each digital image are varied based on the degree of importance, interest or attractiveness of the image, determined as by itself or as related to the group of digital images.
From another aspect, the invention may be seen (a) as a method for varying a printing parameter, e.g., the number of prints, based on a determination of the appeal of the image with respect to certain self-salient features, wherein appeal is an assessment of the degree of importance, interest or attractiveness of an individual image or (b) as a method for varying the same printing parameter for a digital image based on a determination of the emphasis of an image with respect to certain features, wherein emphasis is an assessment of the degree of importance, interest or attractiveness of an individual image relative to other images in a group of images. From this perspective, for both appeal and emphasis assessment, self-salient image features are calculated, such as:
a. People related features: the presence or absence of people, the amount of skin or face area and the extent of close-up based on face size.
b. Objective features: the colorfulness and sharpness of the image.
c. Subject related features: the size of main subject and the goodness of composition based on main subject mapping.
While the above-noted features are adequate for emphasis assessment, it is preferable that certain additional relative-salient image features are considered for appeal assessment, such as:
a. The representative value of each image in terms of color content.
b. The uniqueness of the picture aspect format of each image.
An assessment of an image is obtained with a reasoning engine, such as a Bayesian network, which accepts as input the above-noted features and is trained to generate image assessment values. This assessment may be an intrinsic assessment for individual images, in which case the self-salient features are processed by a Bayesian network trained to generate the image appeal values, or the assessment may be a relative assessment for a group of images, in which case the self-salient and, optionally, the relative-salient features are processed by a Bayesian network trained to generate image emphasis values.
The advantage of the invention lies in its ability to perform an assessment of one or more images and accordingly vary an attribute value, such as, but not limited to, the number of prints, the size of the prints, or the magnification of the prints, made of a digital image without human intervention. In a variety of applications that deal with a group of pictures, such an algorithmic assessment enables the automatic control of image processing, so that the images can be more efficiently processed and printed according to their rank order.